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Abstract —We study variable-length feedback (VLF) codes 
under a strict delay constraint to maximize their average trans¬ 
mission rate (ATR) in a discrete memoryless channel (DMC) 
while considering periodic decoding attempts. We first derive a 
lower bound on the maximum achievable ATR, and confirm that 
the VLF code can outperform non-feedback codes with a larger 
delay constraint. We show that for a given decoding period, as 
the strict delay constraint, L, increases, the gap between the 
ATR of the VLF code and the DMC capacity scales at most 
on the order of 0{L~^) instead of for non-feedhack 

codes as shown in Polyanskiy et al. [“Channel coding rate in the 
finite hlocklengh regime,” IEEE Trans. Inf. Theory, vol. 56, no. 
5, pp. 2307-2359, May 2010.]. We also develop an approximation 
indicating that, for a given L, the achievable ATR increases as 
the decoding period decreases. 

Index Terms —VLE codes, strict delay constraint, DMC. 


I. Introduction 

T he channel capacity, i.e., the maximum coding rate with 
an arbitrarily small error probability, needs an assumption 
of infinite block-length [1]. However, the block error probabil¬ 
ity (BEP) is non-zero in practice since the block-length must 
be finite. Eor given block-length and BEP constraints, lower 
and upper bounds on the maximum coding rate were found 
in [2]. Meanwhile, feedback is known to be very useful for 
performance improvement if variable-length codes are applied. 
In the regime of asymptotically long average block-length, the 
error exponent, regarded as the exponential rate of decay of 
the BEP with respect to the average block-length, has been 
an important performance measure for variable-length codes 
with feedback [3]-[9]. However, the analysis cannot present 
the maximum coding rate for fixed BEP and fixed average 
block-length. 

Recently, Polyanskiy et al. [10] focused on the regime of 
fixed BEP and fixed average block-length in their study of 
variable-length feedback (VLE) codes over discrete memory¬ 
less channels (DMC), formulating the following problem: 

maxM s.t. Pt[W ^W]<e, E[t] < L, (1) 

where M, W, W, and r denote the number of codewords, 
transmitted message, estimated message, and the length of a 

Copyright ©2015 IEEE. Personal use of this material is permitted. Permis¬ 
sion from IEEE must be obtained for all other uses, in any current or future 
media, including reprinting/republishing this material for advertising or pro¬ 
motional purposes, creating new collective works, for resale or redistribution 
to servers or lists, or reuse of any copyrighted component of this work in 
other works. 

This manuscript has been accepted for publication in IEEE Communica¬ 
tions Letters. 

S. H. Kim and T. Le-Ngoc are with the Department of Electrical 
& Computer Engineering, McGill University, Canada (E-mail: seongh- 
wan.kim@mcgill.ca; tho.le-ngoc@mcgill.ca). D. K. Sung is with the Depart¬ 
ment of EE, KAIST, Korea (dksung@ee.kaist.ac.ki'). 

Digital Object Identifier 10.1109/LCOMM.2015.2398866 


received sequence. The positive real values e and L denote 
constraints. A transmitter sends a codeword as time-domain 
symbols and a receiver attempts to decode it at every symbol 
reception and sends a stop-feedback to the transmitter when 
the receiver decides to decode it. Therefore, r also represents 
a delay. They showed that a significant spectral efficiency gain 
can be obtained by using VLE codes for given average delay 
and BEP constraints as compared to the non-feedback fixed- 
length codes. They also considered a strict delay constraint in 
formulating the following problem: 

maxM s.t. Pr[{lK ^ W} U {t > L}] = e. (2) 

Analyzing (Ell, they alleged that the strict delay constraint 
nullifies the gain of VLE codes over the non-feedback fixed- 
length codes. However, problem (El does not maximize the 
average transmission rate (ATR), , which is also defined 

as the spectral efficiency. Therefore, it is not certain whether 
VLE codes have a gain or not over the non-feedback codes in 
terms of the spectral efficiency, under a strict delay constraint. 
Besides VLE codes, they also studied VLE with termination 
(VLET) codes in which the transmitter decides to stop trans¬ 
mitting a message by observing the output of the decoder 
fed back from the receiver and sends a termination signal 
to the receiver under an assumption of error-free feedback 
and feed-forward channels. Chen et al. [11] added practical 
constrains to Polyanskiy’s VLET codes by limiting the length 
of codewords and attempting to decode a codeword only 
at specified times. In their setting, the same codeword is 
reused if the receiver fails to decode it after receiving the 
whole original codeword while the average delay is limited 
as given in problem ([TJ. However, the VLET codes are far 
from practical because existing feedback codes such as Hybrid 
Automatic Repeat & reQuest (HARQ) schemes only use 
acknowledgement (ACK)/negative-ACK (NACK) feedbacks. 

In this letter, we revisit VLE codes under a strict delay 
constraint and a fixed BEP to maximize the achievable ATR 
(instead of the number of codewords). The regime of strict 
(rather than average) delay constraints is desired to guaran¬ 
tee delay requirements of real-time traffic applications. We 
also consider periodic decoding attempts where the receiver 
attempts to decode a codeword periodically. We derive the 
achievable ATR as the lower bound on the maximum ATR 
of VLE codes. We prove that gap between the ATR of VLF 
codes and the DMC capacity scales at most on the order of 
0{L~^) for a given decoding period instead of for 

non-feedback codes with fixed length of L as shown in [2]. 
This comparison shows a significant gain of VLF codes over 
the non-feedback codes under a strict delay constraint. We 
also derive an approximation of the achievable ATR expression 
indicating that, for a given L, the achievable ATR increases 
as the decoding period decreases. In [12], we studied HARQ 



2 


schemes under a strict delay constraint in AWGN channels 
for a given BEP with the following two ideal assumptions: 1) 
the decoding period is sufficiently long; 2) the NACK event 
is identical to the error event. In this letter, without the above 
two assumptions, we find the theoretical bound of VLF codes 
in DMC. 

Throughout this letter, we use the following notation: X, 
X, and Px denote a random variable, its sample value, 
and the probability distribution of X, respectively, = 
{xi,X 2 , ■■■,Xn) denotes an n-dimensional vector and Xj the 
jth element of x”. log(a;) = logg(a:) unless otherwise stated. 


II. Problem Statement 


In this section, we briefly introduce the channel and VLF 
codes which are modified from those in [10] for defining 
periodic decoding attempts, and formulate our optimization 
problem. 

Channel: A DMC consists of a pair of input X and output 
Y on the finite alphabets A and B, respectively, with a 
conditional probability, PYi\Xi = > 1. 

{l,d) VLF code: An (Z,d) VLF code with M messages, 
maximum allowable number of decoding attempts, I, and 
decoding period (i.e., interval between 2 consecutive decoding 
attempts), d, is defined as follows: 

1) A random variable U €lJ with a probability distribution 
of Pu represents a codebook shared by both transmitter 
and receiver. 

2) A sequence of encoders /„ : U x {1, ■■■, M} —> A 
represent the channel input at time n, Xn = fn{U,W) 
where IFs{l,---,M}is the equi-probable message. 

3) A sequence of decoders :Ux -A {1, • • • , M} at¬ 
tempting to provide the estimate of W at time dk where 
k denote the number of decoding attempts, respectively. 

4) A final decision is made by the receiver at a stopping 

time dr* :W = (C/, 

Optimization problem: For an (Z, d) VLF code with 0 < 
e < 1, we maximize its achievable ATR under a strict delay 
constraint as follows: 


r;(z,d,e) 


= max 
M 


logM 

dEpJ 


Pr[W ^Wj<e, 
Pr[r* < Z] = 1. 


L = dl represents the limited length of the received sequence 
(i.e., the delay). Setting cZ = 1 indicates the case of attempting 
to decode a codeword at every symbol reception. 


III. Achievability Analysis 

For achievability analysis, we specify the codebook, en¬ 
coder, and decoder which are modified from those in [10] by 
adding functions to satisfy strict delay constraints as follows: 
Codebook: A codebook U is defined on space U such as 

U = A‘^^x---xA’^\ Pu = Px X ■■■ X Px\ 

Mtimes Mtimes 

where X is distributed according to Px on A. The realization 
of U defines M cZZ-dimensional vectors S A‘^\j = 
!,■■■ ,M. 


Encoder: The encoding sequence /„ maps an equi-probable 
message j to G A’^^ and provides the channel input at time 
n. 


fnU) = {Cj)n for 1 < n < dl, 

where (Cj)„ is the nth coordinate of the vector Cj. 

Decoder: A decoder computes the j-th information density 
at the fc-th decoding attempt for 1 < Zc < Z, 

S,,k = t{C,{dk),Y‘‘>^), j = 


where Cj (n) is the first n symbols of Cj and, the information 
density between a;" and y” is defined as 


z(x";y")=log 


dPY^lx4y"\x^) 

cZPyn (y") 


The decoder defines M stopping instances computing the 
information densities 


A 



inf{0 <k <l : Sj^k > 7 } if V {Sj^k > 7 } 

k—1 

l + 1 Otherwise, 


where 7 is the threshold to stop computing information density 
for each codeword, and for statements A and B, A V B (resp., 
A A B) is true if A or B (resp., A and B) are true. Since 

i 

A {S'j.fc < 7 } can be checked at time Z, a mapping r, = Z -f 1 

k—1 

does not mean that the observation at the (Z + l)-th decoding 
attempt is required. Define a random variable 

t' = min{ri, • • • ,tm}. (3) 


The final decision is made by the decoder at the stopping time 


* ^ J r' if t' < I 

[^ Z if r' = Z -f 1, 


(4) 


i.e., T* is always smaller than or equal to Z. The output of the 
decoder is 


^(vdT-^ A / max{j : t, = r'} if t' < Z 

> \ M if t' = 1 + 1. 


(5) 


For the VLF code specified above, we And a lower bound 
on Tf{l,d,e). 

Lemma 1: For an arbitrary DMC with capacity C, interval 
d, and exp{—C'dZ/2} < a < e, the maximum M satisfying 
Pr[IV 7 ^ W] < e and r* < Z is lower-bounded by 

Proof: Let us define the following instances: 

T = inf{fc > 1 : i{X'^'^- Y'^'^) > 7}, (6a) 

f = inf{fc > 1 : i{X^'^-, > 7}, (6b) 


maxM > 

PrlW^W]<e 


where X and X are independent random variables with 
the same distribution and Y is the channel output when X 
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is channel input. The average probability of error with M 
codewords is upper-bounded as 


T>[g{Y<^^ )^W]< ¥[g{Y‘‘^ ) ^ 1|VF = 1] 

M 

y {Tj < Ti < /} [J{1 < Ti} IT" = 1 
_j=2 

< (M - l)P[r2 < ri < ;|1T" = 1] + ¥[l < ti|1T 


= (M - l)P[f <T <l]+¥[l <t] 
<{M - l)P[f < t]+P[1 < t], 


(7) 

( 8 ) 

1] (9) 
( 10 ) 
( 11 ) 


where (|7]i follows from Q, (HI) from ([^l, ( 01 ), and (O, (O from 
a union bound and symmetry for Tj, (fTOl i from (l 6 ]l. 

Note that, as compared to (45) in [10], (E]) contains the 
additional term P[^ < r] for the event that no codeword 
is detected within I decoding atternpts. Plf < t] is upper- 
bounded by (see (111)-(118) of [lolj 


Proof: 


]E[r*] < min(E[T^],/) 

= min(E[Ti|Vl" = 1],/) 
= min (E[t], /) , 


(17) 

(18) 

(19) 

( 20 ) 


where dnii follows from dH), (O follows from dll), (O 
follows from symmetry, and (l20l i follows from (l 6 al) . Since 
ydfe) _ Y) = ^ martin¬ 

gale, from Doob’s optional stopping theorem [10], we obtain 
E[i(X‘^'", — drC] = 0. Then, E[r] is upper-bounded as 


E[t] 


¥[i{X<^\Y<^^)] ^ 7 (a) -fdao 
dC - dC 


( 21 ) 


IPIt < t] < exp{— 7 }. 
P[Z < t] is upper-bounded by 

r !■ 


P[l < t] 


< 7 } 


_fc=l 

< P [i{X‘^^;Y'^^) < 7 ] 

= P[i{X‘^^;Y'^^) < {l-S)Cdl] 

< exp{—S^Cdl/2}, 


( 12 ) 

(13) 

(14) 

(15) 


where dTH) follows from d^ . ([T4li is obtained by setting 7 = 
(1 — S)Cdl for C = E[i( 2 f;y)] and 0 < b < 1, and (ITSI l is 
obtained by using a Chernoff bound. 

Instead ofP[g{Y‘^'^ ) f W] < e, we set a stricter constraint 
using dni) and substitute dni) and dUll into dni), 

(M — 1) exp{ —(1 — S)Cdl} + exp {—S^Cdl/2} < e. (16) 

Notice that any M satisfying dTbl l also satisfies P[g(Y'^'^ ) 
W\ < e. To satisfy dT^ . let exp{—5'^Cdl/2} = a for 0 < a < 
e. For 0 < (5 < 1, the range of a becomes exp{—Cdl/2} < 
a < e. Therefore, dTSI l can be rewritten as 

M < (, - „)exp I (^1 - Cdij + 1 


where (EB follows from dbaji and 7 (a) = 

Cdl. Note that (i20l) is similar to 


1 - 


21og(l/a) 


Cdl 


(39) of [ 10 ] except for the limitation to I and (i2Tl i is similar 
to (107) of [10] except for d and 7 (a). ■ 

Theorem 1: For an arbitrary DMC with capacity C, and 

Cdl 

e 2 < e < 1 , 


Tf*(Td, e)> max 

J ^ ^ ^ ' — ndi 


log(e — a) -I- Cdl — y^2Cdl log(l/a) 


<“<'^dmin 


1 - 


2Iog(l/a) 

Cdl 


1 + ^,1 


( 22 ) 


Proof: From Lemmas 1 and 2, 

logM ^ _ logM(a) 


max 


> 


M dE[T*|M] ^-Sf. 


max 


<a<e 


dE[r*|M(a)] 


log 


> max 

_ Cdl 


(e - a) exp |(^1 - Cdl^ + 1 


' <a<e 


1 - 


21og(l/a) 

Cdl 


> max 
- 

e ^<a<e ^ 


l+f,l 

log(e — a) -I- Cdl — ^j2Cdl log(l/a) 


1 - 


21og(l/a) 

Cdl 


1+^,1 


for exp{—C'cd/2} < a < e. We know that M{a) = 
[(e — a) exp{(l — \J ^^°^^j ^)Cdl } -|- IJ is the maximum M 
satisfying dTbl) . In other words, we can achieve M{a) with 
Pr[lF fW]<e and Pr[T* < (]. ■ 

Lemma 2: For an arbitrary DMC with capacity C, interval 
d, and exp{—CdZ/2} < a < e, E[r*] is upper-bounded as 

where oq is the maximum value of i{X, Y). 

^Polyanskiy et al. proved GD only for d = 1 . However, the proof is easily 
extended to the cases of d > 1. 


While Theorem 1 offers the expression to compute the lower 
bound on the maximum ATR in terms of C,f d, and e, the 
expression does not provide a simple view of the effect of I on 
Tf{l,d,e)- The following theorem establishes the asymptotic 
behavior of the gap between C and 7^*((, d, e) as I increases. 

Theorem 2: For an arbitrary DMC with capacity C and 
any e~^~ < e < 1 , the gap between the capacity and the 
maximum ATR of a VLF code scales at most, 

AvLF = C'-r/(/,d,e) =0(1/0, 

where f{n) = 0{g{n)) iff there are constants c and uq such 
that f{n) < cg{n) Vn > tiq. 
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Proof: Avlf = C — max is upper-bounded as 


Avlf < uiin C — 


log(e - a) -b Cdl - y/2Cdl log(l/a) 


< 


dao — log(e — a) 


-^<c<n.in{.,e-^} 1 - J ] dl + ^ 


(23) 
, (24) 


where (l2?t follows from Theorem 1 and (l24l) follows from 
limiting the range of a by eliminating min in the denominator 
and from the relationship, /3o - P 1 /P 2 = (/32 - Idilh- 
For terms related to I on the right-hand-side (RHS) of (l24l i. 
we obtain the following three convergences: 


lim exp {—Cdl/2} = 0, lim exp {—ald/{2Cl)} = 1, 

I—foe l—foo ^ ^ 


lim 

l—^oo 


dl - y/2dllog{l/a)/C + dao/C 

I 


= d. 


Therefore, there exist constants li and I 2 such that 
exp { —^} < f for any I > li and | < exp{ —for any 
I > h, respectively. In addition, by setting a = e/2, for a given 
constant 0 < Cq < 1 , there exists a constant Z 3 > max{Zi,Z 2 } 

such that - ^ > Cod for any I > I3. Finally, 

we have for I > I 3 


Av 


< 


dao — log(0.5e) 


dl - 


< 


2dl log(2/e) . dao 

C C 


dao — log(0.5e) 
Idco 


(25) 


Therefore, Avlf = 0(1/0- ® 

Remarks: 

1) For given d, since L is linearly proportional to I, Avlf 
scales as Avlf = 0 ( 1 /L) from Theorem 2 . 

2) From (l25l l. the achievable ATR expression is approxi¬ 
mated for L ^ d as 

( 26 ) 

L 

3) From (l26l l. for a given L, the approximation of 
Tf{l,d,e) increases as d decreases. 


IV. Illustrative Results 

We show some illustrative results obtained from the analysis 
in Section ini] in terms of ATR [b/s/Hz]. To use b/s/Hz as a 
unit of measure in illustrative results, quantities related with 
the spectral efficiency are divided by logg 2 . 

Fig.IUplots the ATR [b/s/Hz] versus L for binary symmetric 
channel (BSC) with crossover probability q = 0.11, e = 10“^, 
and C = 0.5 b/s/Hz for d = 1, 50, and 100. In BSC(g) channel, 
C = log(2) - H{q) and oq = max{log(2g), log(2(l - q))}. 
The legend ‘No feedback’ corresponds to the fixed blocklength 
codes without feedback given by Theorem 52 of [2]. It is 
interesting to observe that the ATR of VLF codes converges 
to the capacity faster than that of the fixed-length codes even 
under a strict delay constraint. When d = 1, to achieve 90% 
of the capacity, L = 360 is enough for VLF-code while 
at least 3100 is required for non-feedback code. The ATRs 



Fig. 1. The ATR vs. L for the BSC(O.ll); probability of error e = 10 ® for 
d = 1, 50, and 100. 


of the VLF code are lower than those of the non-feedback 
code for L < 120 with d = 1, L < 450 with d = 50, 
and L < 1200 with d = 100. In VLF codes, decoding 
attempts at k < I give an opportunity to stop the transmission 
before 1. However, these attempts simultaneously increase the 
probability that other messages than the transmitted one are 
detected as an estimated message. As L increases, the positive 
effect of VLF codes dominates the negative effect. Therefore, 
as L increases, VLF codes offer much higher ATR than non¬ 
feedback codes. We can also observe that, as d decreases, 
the ATR increases for a given L as indicated by (l26l l. In 
practice, smaller d may need faster processing or incur higher 
complexity, which should be considered in the design of VLF 
codes for a specific application. As shown in Fig. [H the 
approximation (| 2 ^ provides results very close to those of (l 2 ^ . 

V. Conclusions 

We investigated a VLF code under a strict delay constraint 
to maximize the ATR in DMCs while considering periodic 
decoding attempts. We first derived a lower bound on the 
maximum achievable ATR and showed that the VLF code can 
outperform the non-feedback codes for increasing L. We also 
proved that the gap between the ATR of the VLF code and 
the DMC capacity scales at most on the order of 0{L~^) 
for a given d instead of for non-feedback codes 

as previously shown in [2]. The approximated expression of 
the ATR indicates that the ATR increases with decreasing 
decoding period d. 
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